Imaging system and imaging method

ABSTRACT

An imaging system  100  for imaging a sample in a medium carried in a container WP as an imaging object comprises: an imager  21  which obtains an original image by imaging the imaging object; and a data processor  33  which generates multi-gradation image data by performing a gradation correction on the original image, wherein the data processor  33  associates a luminance value corresponding to a luminance of the medium in the original image with a maximum gradation value in the gradation correction.

CROSS REFERENCE TO RELATED APPLICATION

The disclosure of Japanese Patent Application No. 2013-187116 filed onSep. 10, 2013 including specification, drawings and claims isincorporated herein by reference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to an imaging system and an imaging method forimaging a sample in a medium carried in a container as an imagingobject.

2. Description of the Related Art

In medical and bio-scientific experiments, liquid or gel-like fluid(e.g. culture solution, medium or the like) is poured into each well ofa plate-like tool (e.g. called microplate, microtiter plate or the like)in which a multitude of recesses also called as wells are, for example,arrayed, and things cultured herein such as cells are observed andmeasured as samples. In recent years, samples have been imaged anddigitized by a CCD camera or the like and various image analysistechniques have been applied to image data for observation and analysis.

In an imaging system of this type, a correlation between actual opticaldensities of imaging objects and gradation values obtained by expressingthe optical densities by multi-gradation image data is not necessarilylinear due to nonlinearity in the sensitivity of an, imaging system.Thus, a gradation correction (also called as a gamma correction or atone correction) to properly adjust this is necessary. A techniqueapplied to a document scanner for optically reading a plane document isknown as a precedent of such a gradation correction technique. Forexample, in an image processing apparatus described in JP2003-209693A,pre-scanning is performed with a low resolution prior to actual scanningin scanning and reading a document image. A correction characteristicfor a gradation correction, specifically a correlation between inputgradation values and output gradation values is obtained from ahistogram distribution characteristic of an obtained image and agradation correction processing is performed during actual scanning.

It is considered to apply a gradation correction technique like theconventional technique also to an imaging system used for the purpose ofobserving samples such as cells. If imaging objects are samples in aculture medium, the medium is not perfectly transparent and has anoptical density of a certain degree. Thus, there is a characteristicthat a luminance of each pixel of an imaged image is not higher than aluminance corresponding to the medium itself. However, such a problem isnaturally not considered in the above conventional technique aiming toread a document and only a correlation between luminance values andgradation values at intermediate gradations is focused. Thus, if thisconventional technique is directly applied to the imaging of biologicalsamples, gradation values are assigned up to a luminance level which isnot applicable in an actual image, wherefore there has been a problemthat a range of invalid gradation values not used in the multi-gradationexpression of an image is created and a dynamic range of densityexpression in a multi-gradation image data is limited.

SUMMARY OF THE INVENTION

This invention was developed in view of the above problem and aims toprovide a technique capable of generating multi-gradation image datarepresenting an image obtained by imaging a sample in a medium in a widedynamic range.

An imaging system for imaging a sample in a medium carried in acontainer as an imaging object according to the present inventioncomprises: an imager which obtains an original image by imaging theimaging object; and a data processor which generates multi-gradationimage data by performing a gradation correction on the original image,wherein the data processor associates a luminance value corresponding toa luminance of the medium in the original image with a maximum gradationvalue in the gradation correction.

An imaging method for imaging a sample in a medium carried in acontainer as an imaging object according to the present inventioncomprises: an imaging step of obtaining an original image by imaging theimaging object; and a data processing step of generating multi-gradationimage data by performing a gradation correction on the original image,wherein the multi-gradation image data is generated so that a luminancevalue corresponding to a luminance of the medium in the original imageis associated with a maximum gradation value in the gradation correctionof the data processing step.

In these inventions, in view of the above characteristic in the case ofimaging the sample in the medium as the imaging object, themulti-gradation image data is generated based on a relationship betweenthe luminance values and the gradation values associated such that theluminance value corresponding to the luminance of the medium providesthe maximum gradation value. That is, the luminance values and thegradation values are associated in a luminance range having theluminance of the medium as an upper limit, and the original image isexpressed in the form of the multi-gradation image data based on that.Thus, the gradation values are assigned according to the luminance rangeof an actual image of the imaging object and the image can be expressedin multiple gradations effectively using a dynamic range of thegradation values.

According to the present invention, the gradation correction processingis performed with the luminance values and the gradation valuesassociated such that the luminance value corresponding to the luminanceof the medium provides the maximum gradation value. Thus,multi-gradation image data representing an image obtained by imaging abiological sample in a medium in a wide dynamic range can be obtained bypreventing the creation of a range of invalid gradation values.

The above and further objects and novel features of the invention willmore fully appear from the following detailed description when the sameis read in connection with the accompanying drawing. It is to beexpressly understood, however, that the drawing is for purpose ofillustration only and is not intended as a definition of the limits ofthe invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view diagrammatically showing the configuration of oneembodiment of an imaging system according to the invention.

FIGS. 2A and 2B are a graph and a view outlining the gradationcorrection processing.

FIGS. 3A and 3B are views diagrammatically showing samples carried inthe well plate.

FIG. 4 is a graph illustrating the scaling of the gradation correctioncharacteristic.

FIG. 5 is a flow chart showing the first mode of the imaging operationof this imaging system.

FIG. 6 is a flow chart showing the operation of a calibration process.

FIG. 7 is a flow chart showing the second mode of the imaging operationof this imaging system.

FIG. 8 is a diagram showing an example of the reference table.

FIGS. 9A and 9B are views showing a difference between images due to thepresence or absence of the adjustment of the tone curve.

FIGS. 10A and 10B are views showing an arrangement example of samples onthe well plate.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1 is a view diagrammatically showing the configuration of oneembodiment of an imaging system according to the invention. This imagingsystem 100 is a so-called line CCD scanner apparatus using a CCD linesensor as an imaging device and includes an imaging unit 1 with a sampleholder part 10, an optical scanner part 20 and a control part 30, and ahost computer 50.

The sample holder part 10 includes a holder 11 for holding a well plateWP substantially in a horizontal posture by being held in contact with aperipheral edge part of the lower surface of the well plate WP. Wells Wfor carrying a culture medium containing biological samples such ascells as imaging objects are formed on the upper surface of the wellplate WP. Further, the sample holder part 10 includes a white referenceplate 12 and an AF reference plate 13 to be read as references in ashading correction processing and an autofocus (AF) adjustmentprocessing respectively to be described later.

The well plate WP includes a plurality of, e.g. ninety six (12×8 matrixarray) wells W each having a substantially circular cross-section andcapable of carrying a liquid or solid medium. A diameter and a depth ofeach well W are typically about several millimeters. Note that the sizeof the well plate and the number of the wells for this imaging system100 are not limited to these, and are arbitrary. For example, there maybe 384 wells.

When the well plate WP is placed on the holder 11 in a state where theculture medium containing the samples is held in these wells W, light(e.g. white light) is irradiated to the well plate WP from a lightsource 29. The light source 29 is, for example, an LED lamp and arrangedabove the well plate WP held on the holder 11.

The optical scanner part 20 functions as an imager for optically imagingthe imaging objects by receiving transmitted light from the imagingobjects. The optical scanner part 20 includes an imaging part 21 inwhich CCD elements 22 as light receiving elements and a convergentoptical system 23 for adjusting a magnification of an optical image bytransmitted light are arranged below the well plate WP, a focusingmechanism 24 for focusing the convergent optical system 23, and a scandrive mechanism 25 for driving the imaging part 21 in a predetermineddirection (lateral direction in FIG. 1), for example, by belt drive.

Out of the light irradiated toward the well plate WP from the lightsource 29, light transmitted downward from the bottom surface of thewell W is converged by the convergent optical system 23 and received bythe CCD elements 22, whereby an optical image is converted intoelectrical signals. The focusing mechanism 24 drives the convergentoptical system 23 in response to a control command from the control part30, thereby adjusting a focus position of the optical image focused onthe CCD elements 22. Further, the scan drive mechanism 25 moves thelight source 29 and the imaging part 21 integrally in a horizontalplane. Thus, a positional relationship between the light source 29 andthe imaging part 21 is fixed.

Scanning movements of the CCD elements 22 relative to the well plate WPare realized by moving the CCD elements 22, which constitute a linesensor, in a direction perpendicular to an arrangement direction of theCCD elements 22 relative to the well plate WP. In this way, atwo-dimensional image of the biological samples as the content of thewell W is imaged. The optical scanner part 20 is controlled by thecontrol part 30.

The control part 30 includes an A/D converter 31, a memory 32, a CPU 33and a driver 34. The A/D converter 31 converts electrical signals outputfrom the CCD elements 22 into luminance values (color density values)according to the received amounts of the transmitted light from thesamples in the well W. The memory 32 holds a collection of the luminancevalues of respective pixels obtained from the samples as image data andstores various setting data. The CPU 33 functions as a controller forcontrolling each component of the apparatus. Further, the driver 34drives the focusing mechanism 24 and the scan drive mechanism 25 inresponse to a control command from the CPU 33. The memory 32 is composedof a ROM, a RAM or a nonvolatile memory and the like, and includes abuffer memory 32 a for temporarily holding luminance value data outputfrom the A/D converter 31 and an image memory 32 b for holdingmulti-gradation image data generated based on the luminance value data.Besides these, various pieces of reference data such as a referencetable (LUT) 32 c to be referred to in performing a gradation correctionprocessing to be described later is stored in the memory 32 in advance.

The control part 30 thus configured can communicate with the hostcomputer 50 for controlling the operation of the entire imaging system100 via an interface part (I/F) 35. Specifically, the host computer 50is configured similarly to a general personal computer and includes aCPU 51 for performing various arithmetic processings, a memory 52 fortemporarily saving control data generated by the operation of the CPU51, a storage 53 storing a control program to be executed by the CPU 51,an interface part (I/F) 54 for data transfer to and from the controlpart 30.

The host computer 50 also includes a user interface (UI) part 55 forreceiving the input of various operations from a user and presentingvarious pieces of information to the user. More specifically, the UIpart 55 includes at least one type of an input device such as operationbuttons, a keyboard, a mouse or a touch panel as a receiver forreceiving operational inputs from the user. Further, the UI part 55includes a display for displaying, for example, obtained images,messages and the like on a screen.

Functions of receiving the input of various operations from the user tooperate the imaging system 100 and presenting images obtained as aresult of operations to the user are consolidated into the host computer50. Accordingly, the control part 30 has only a minimum configurationfor causing the optical scanner part 20 to perform a predeterminedoperation. As just described, the control part 30 having controlfunctions minimum necessary to operate particular hardware is providedin this apparatus, whereas more general processings are performed by theversatile host computer 50, whereby system cost can be suppressed low.

Note that this imaging system 100 is composed of the imaging unit 1integrally configured by the sample holder part 10, the optical scannerpart 20 and the control part 30, and the general-purpose host computer50 for controlling this imaging unit 1 as described above. Instead ofthis, all components necessary for imaging may be integrallyincorporated. In the case of an imaging system composed of an imagingpart and a host computer, hardware and software resources for storingimage data and performing various analysis processings can beconsolidated into the host computer. Since this makes it sufficient toinclude only hardware and software minimum necessary for imaging in theimaging unit, system cost can be suppressed.

Next, the operation of the imaging system 100 configured as describedabove is described. When receiving an instruction to start an imagingoperation, i.e. an operation of reading the samples held in the sampleholder part 10 from the user, this imaging system 100 images samples byperforming a reading operation under designated conditions. Imagesignals generated by the CCD elements 22 of the imaging part 21 by thereading are converted into multi-gradation original image data by theA/D converter 31 of the control part 30. Original image data at thistime is affected by a nonlinear sensitivity characteristic of theimaging system. Thus, this imaging system 100 performs a gradationcorrection processing for the original image data to generatemulti-gradation image data having such nonlinearity removed. Thefollowing data processings are performed for each of color-separatedcolor components in a system for imaging a color image.

FIGS. 2A and 2B are a graph and a view outlining the gradationcorrection processing. More specifically, FIG. 2A is a graph showing anexample of a gradation correction characteristic and FIG. 2B is a viewdiagrammatically showing a data flow of the gradation correctionprocessing in this imaging system 100. In the case of expressingmulti-gradation image data, for example, by 8-bit data, the number ofgradations is 256 and a gradation value takes a value of 0 to 255 asshown in FIG. 2A. Light with highest luminance received by the CCDelements 22 is expressed by a luminance value of 100%, a zero lightquantity is expressed by a luminance value of 0%, and a gradation valueof 0 and a gradation value of 255 are respectively assigned to theluminance value of 0% and the luminance value of 100%. As shown in FIG.2A, nonlinearity in the sensitivity of the imaging system is compensatedby providing a linear correlation between the luminance values and thegradation values, whereby multi-gradation image data truer to an opticalcharacteristic of imaging objects is obtained.

In this imaging system 100, such a correlation between the luminancevalues and the gradation values is digitized and stored as the look-uptable (LUT) 32 c in advance. As shown in FIG. 2B, analog image signalsoutput from the CCD elements 22 are converted into digital data(original image data) by the A/D converter 31 and temporarily saved inthe buffer 32 a. Then, the CPU 33 refers to the LUT 32 c based on theoriginal image data output from the buffer 32 a, converts the originalimage data into multi-gradation data having nonlinearity in thesensitivity of the imaging system corrected and stores themulti-gradation data in the image memory 32 b. In this way, thegradation correction processing is performed.

Here, a data length of the multi-gradation image data finally stored inthe image memory 32 b is eight bits. On the other hand, the originalimage data handled by the A/D converter 31 and the buffer 32 a has alonger bit length, e.g. twelve to sixteen bits. Further, an effectivebit length in computations in the CPU 33 is sufficiently longer thaneight bits. In such a configuration, the degradation of image qualitycan be prevented by suppressing the generation of noise in a process forgenerating multi-gradation image data from image signals.

Further, the image signals output from the CCD elements 22 aresuccessively converted into digital data and saved in the image memory32 b after the gradation correction processing is performed thereon.That is, the original image data output from the A/D converter 31 isonly temporarily saved in the buffer 32 a for the correction processing,and the original image data corresponding to the entire image of onewell plate WP or one well W is not saved. Thus, the image memory 32 bhas only to have a function and a capacity to save only 8-bit data afterthe gradation correction and the buffer 32 a may have a small capacity.

Note that the buffer 32 a is described as an independent function blockto show the flow of the process in this specification, the buffer 32 acan be realized, for example, by utilizing an internal buffer of a CPUor a part of a memory space of an image memory as a buffer area in anactual apparatus. For example, if image data after the gradationcorrection is transferred to the host computer 50 and stored in the hostcomputer 50, a capacity of the image memory necessary in the imagingunit 1 can be largely reduced.

Next, how to set a light quantity equivalent to the luminance value of100% in the original image data is described. In an imaging apparatus ofthis type, a shading correction of obtaining a shading characteristic byimaging a reference surface with a known optical characteristic andnormalizing an incident light quantity based on this characteristic isgenerally performed to correct a sensitivity variation of an imagingdevice. Although specific processing contents are described in detaillater, a shading correction processing is performed based on an imagingresult of the white reference plate 12 having a predeterminedtransmittance also in this imaging system 100. Since there are variousknown techniques for the shading correction processing and a techniqueappropriately selected therefrom can be applied also to this imagingsystem 100, detailed description is not given here.

However, apparatuses for imaging samples such as cells carried togetherwith a culture medium in the wells W of the well plate WP like thisimaging system 100 have the following characteristics specific to suchimaging.

FIGS. 3A and 3B are views diagrammatically showing samples carried inthe well plate. More specifically, FIG. 3A is a side sectional view ofthe well plate WP carrying a culture medium containing biologicalsamples, and FIG. 3B is a bottom view of FIG. 3A. Imaging objects inthis imaging system 100 are biological samples such as cell clusters (orspheroids) Sp present in the liquid, solid or gel-like medium (culturemedium) M carried in the wells W provided in the well plate WP as shownin FIG. 3A. Here, although the cell clusters Sp distributed on the innerbottom surface of the well W are illustrated as an example, the imagingobjects may be present on the surface or an intermediate part of themedium M. Further, the amount of the medium M (depth in the well W) alsovaries.

The medium M contains certain drug in many cases and is rarely aperfectly transparent body or white body. Accordingly, in the case ofobserving the imaging objects from below, the imaging objects such asthe cell clusters Sp are generally distributed in the medium M having acertain optical density as shown in FIG. 3B. Thus, in an image obtainedby imaging the interior of the well W, a luminance value of the medium Mis highest and other parts have a lower luminance than this. That is, anactual image has a luminance distribution having the luminance valuecorresponding to the medium M as a maximum value.

On the other hand, since the luminance values are normalized based onthe white reference plate 12 having a constant optical characteristic inthe shading correction processing, a normalization result does notnecessarily match a luminance distribution range of an actual imageobtained by imaging the well W. Accordingly, in this imaging system 100,the gradation correction characteristic is scaled according to theluminance value of the medium M.

FIG. 4 is a graph illustrating the scaling of the gradation correctioncharacteristic. The imaged light quantities (luminance values) are sonormalized that the luminance value obtained by imaging the whitereference plate 12 is 100% in the shading correction processing as shownby a curve A in FIG. 4. In the LUT 32 c for the gradation correctionprocessing, the maximum gradation value (255 in the case of eight bits)corresponds to the luminance value of 100% and the minimum gradationvalue of 0 corresponds to the luminance value of 0%.

On the other hand, in the actual image obtained by imaging the well W,the medium M has the highest luminance as described above. The shadingcorrection processing is so performed that the luminance value is nothigher than 100% regardless of samples imaged. The luminance value ofthe medium M varies depending on the type and state of the medium M.Accordingly, in many cases, the luminance value of the medium M is lowerthan 100%. If the luminance value of the medium M in the actual image isX % as shown in FIG. 4, multi-gradation image data corresponding to theactual image after the gradation correction is expressed in a range from0 to a gradation value Y corresponding to the luminance value X and anumerical value range above the gradation value Y up to 255 is not usedfor the data representation of the image. That is, some of 256gradations become invalid to narrow a dynamic range of multi-gradationexpression.

Accordingly, in this embodiment, the gradation correction characteristic(curve A) associated with the gradation values in a luminance valuerange of 0% to 100% is scaled in a range from 0% to the luminance valueX % of the medium M to obtain a corrected gradation correctioncharacteristic (curve B) as shown by an arrow in FIG. 4. As a result,the gradation values from the minimum gradation value to the maximumgradation value are assigned in a luminance distribution range of theactual image. By applying the thus corrected gradation correctioncharacteristic to perform the gradation correction processing, theactual image obtained by imaging the well W can be expressed in multiplegradations effectively using 256 gradations. This enables images to beexpressed in a wider dynamic range.

Note that data stored in the LUT 32 c preferably has a bit length longerthan eight bits to prevent a reduction in computation accuracy caused bybit drop-out at the time of scaling. Further, a slight margin may beprovided on a high gradation side in the assignment between theluminance values and the gradation values. Specifically, instead ofassigning the maximum gradation value of 255 to the luminance value ofthe medium M, a slightly smaller gradation value (e.g. 250) may beassigned. By doing so, even a case where there is an area having ahigher luminance than the medium or there is an error in calculating theluminance value of the medium can also be dealt with. In the followingdescription, curves representing the gradation correction characteristiclike the curves A, B shown in FIG. 4 may be referred to as “tonecurves”.

Next, two modes of an operation of imaging biological samples by thisimaging system 100 using the above scaling of the gradation correctioncharacteristic (tone curve) is described. In a first mode describedbelow, the luminance value of the medium M is obtained by actuallyimaging given samples and the scaling is performed based on that value.On the other hand, in a second mode, the scaling is performed based onthe luminance value of the medium M estimated from the type and state ofthe medium M. These two modes can be realized by using the same hardwareconfiguration and making the operation of the hardware configurationpartly different.

<First Mode>

FIG. 5 is a flow chart showing the first mode of the imaging operationof this imaging system. Further, FIG. 6 is a flow chart showing theoperation of a calibration process. These processes are realized by theCPU 33 executing the control program stored in the memory 32 in advanceand causing each component of the apparatus to perform a predeterminedoperation.

When the host computer 50 receives an instruction input to the effect ofstarting the imaging operation from the user via the UI part 55,information (imaging information) designating the contents of theinstruction and various imaging processings included in the userinstruction is obtained by the control part 30 of the imaging unit 1from the host computer 50 (Step S101). The imaging information includes,for example, the number and type of the well plate(s) WP to be imaged,the content of samples carried therein and imaging conditions (e.g.illumination condition, resolution). The imaging unit 1 having receivedthis information performs the calibration process shown in FIG. 6 (StepS102).

The autofocus (AF) adjustment processing (Steps S201 to S204) and aprocessing for obtaining the shading characteristic (shading processing;Steps S205 to S207) are performed in the calibration process of thisembodiment. Known techniques can be applied as the contents of theseprocessings. Accordingly, the principles and detailed operations ofthese processings are not described here.

In the calibration process, the AF adjustment processing (Step S201 toS204) is first performed. Specifically, the scan drive mechanism 25 isactuated to move the imaging part 21 to a position right below the AFreference plate 13 (Step S201) and the optical scanner part 20 reads theAF reference plate 13 (Step S202). A predetermined reference line isdrawn on the AF reference plate 13, an image of the reference line isobtained every time while a focus position of the convergent opticalsystem 23 is changed and set in multiple stages by the focusingmechanism 24. The focus position where an image contrast of thereference line is maximized is set as an in-focus position and thesetting of the convergent optical system 23 at that time is stored as AFdata DA1 (Step S203) and the convergent optical system 23 is adjustedaccording to that setting (Step S204).

In this way, the in-focus position of the imaging part 21 is optimized.The samples are imaged in a state where the convergent optical system 23is adjusted based on the AF data DA1 at this time until the autofocusadjustment processing is performed again.

Subsequently, the shading processing is performed (Steps S205 to S207).Specifically, the scan drive mechanism 25 is actuated to move theimaging part 21 to a position right below the white reference plate 12(Step S205) and the optical scanner part 20 reads the white referenceplate 12 (Step S206). The white reference plate 12 is a white flat platehaving a predetermined light transmittance and transmits a part of lightfrom the light source 29 positioned above the white reference plate 12to allow it to be incident on the optical scanner part 20.

If V denotes a true color density value of the white reference plate 12and an output value when the optical scanner part 20 reads the whitereference plate 12 is shading data DS1, a relationship of:

DC=DD×V/DS1  (Equation 1)

holds for each pixel between read image data DD obtained by reading thesamples and corrected image data DC after the shading correction.

The execution of the shading correction is nothing other than theobtainment of the corrected image data DC from the read image data DDbased on the relationship of (Equation 1). In other words, by performingthe shading processing to obtain the shading data DS1, a correctioncoefficient, i.e. a coefficient (V/DS1) on the right side of(Equation 1) when the shading correction is performed in the subsequentreading operations is obtained. Accordingly, this shading data DS1 isobtained from a reading result and stored (Step S207).

As just described, information necessary to perform the shadingcorrection is obtained by performing the shading processing. Asdescribed later, the shading correction using the coefficient (V/DS1)obtained from the shading data DS1 at this time as a correctioncoefficient is performed until the shading processing is performedagain.

Referring back to FIG. 5, a preparation for actually reading the samplesis set when the calibration process is finished as described above.Subsequently, an internal parameter T indicating the number of theremaining well plates WP to be imaged is set to an initial value ndesignated as the imaging condition by the user (Step S103) and whetheror not an adjustment of the tone curve is necessary is judged (StepS104). Here, the “adjustment of the tone curve” means a processing ofscaling the tone curve stored in the LUT 32 c according to the luminancevalue of the medium such as in the processing of obtaining the curve Bby scaling the curve A shown in FIG. 4.

Whether or not the adjustment of the tone curve is necessary can bejudged, for example, by the following criteria. First, if the samples tobe imaged are new ones that have not been imaged thus far, the grasp ofthe luminance value of the medium M and an associated adjustment of thetone curve are necessary since the luminance value of the medium M isunknown.

On the other hand, if the samples as the imaging objects have beenalready imaged, it is, in principle, not necessary to adjust the tonecurve. In an imaging system for imaging biological samples as imagingobjects, so-called time-lapse imaging may be performed to regularlyimage the same samples at predetermined time intervals to observechanges in the samples with time. In this case, since comparativeobservation of images cannot be precisely performed if the imagingconditions change, imaging needs to be performed under the same imagingconditions. Thus, it is desirable to use the tone curve used in theprevious imaging also in the later imaging and a re-adjustment of thetone curve is not necessary.

Note that even if the same samples are used, the color of the mediumgradually changes with the passage of time and generally becomesgradually darker. Because of this, it is expected that the imagecontrast gradually decreases by continuing to apply the same tone curve.To deal with such a case, a configuration that can re-adjust the tonecurve according to need may be adopted.

Also in the case of changing the imaging condition such as an intensityof illumination light, an imaging resolution or a scanning speed, theshading processing and the tone curve need to be adjusted.

Further, regardless of these requirements, if an instruction to theeffect of making such adjustments or not making them is input by theuser, that instruction is prioritized as a matter of course and whetheror not the adjustment of the tone curve is necessary is judged.

If the adjustment of the tone curve is judged to be necessary, thepre-scanning of the samples and the adjustment of the tone curve basedon this are subsequently performed (Steps S111 to S113). Specifically,the optical scanner part 20 operates for pre-scanning prior to actualscanning to image the well plate WP (Step S111) and the CPU 33calculates the luminance value of an area corresponding to the medium inan obtained image of the well W (Step S112).

Since the imaging in this case is sufficient to be able to merely obtainthe luminance value of the medium M and does not require a highresolution, the scanning speed may be set higher than normal or ascanning region may be limited. However, the shading correctionprocessing based on the previously obtained shading data DS1 isperformed on the imaged image.

As described above, since the luminance value of the medium M is thoughtto be highest even in an image including the cell clusters Sp and thelike, a luminance value of an area with the highest luminance in theimage can be simply regarded as the luminance value of the medium M.More precisely, it is, for example, possible to detect the cell clustersSp and the like from the image by an image processing, set an areaexcluding areas taken up by them as an area corresponding to the mediumand obtain a luminance value of that area. However, it is realistic toperform such a complicated processing in the host computer.

When the luminance value of the medium M is obtained in this way, thetone curve is scaled according to the luminance value of the medium Mbased on the principle shown in FIG. 4. Then, a new correlation betweenthe luminance values of the original image before the correction andgradation values of the image after the correction is obtained (StepS113). The tone curve is adjusted in this way. This causes a correctioncharacteristic applied to the later gradation correction processing tobe determined.

Since the color of the medium carried in the well W differs if the typeand amount thereof differ, the tone curve needs to be originallyadjusted based on the luminance value of the medium M for each well W.However, since the luminance value is thought to be substantially equalif the type and amount of the medium and a preparation time thereof arethe same, a common tone curve may be applied to each well W in such acase. Here, it is assumed that the medium of the same conditions isprepared in the entire well plate WP and a case of including differentmedia is described later.

If the tone curve is adjusted or the adjustment is judged to beunnecessary in Step S104, actual scanning is subsequently performed forthe samples (Step S105). Actual scanning is the scanning of the imagingpart 21 to obtain an image of the samples and imaging is performed witha resolution and under imaging conditions designated by the user. Whenthe original image data DD of each pixel is obtained by actual scanning,the shading correction based on (Equation 1) and the gradationcorrection processing based on the adjusted tone curve are performedpixel by pixel on the original image data DD, whereby image dataexpressing the original image in 256 gradations is obtained (Step S106).This multi-gradation image data is saved in the image memory 32 (StepS107) and utilized in various image processing apparatuses and the likeas an output of the imaging system 100.

When imaging is finished for one well plate WP in a manner as describedabove, the parameter T indicating the number of the remaining wellplates WP is decremented by 1 (Step S108) and the processings in andafter Step S104 are repeated until the value of the parameter T becomes0 (Step S109). At this time, whether or not it is necessary to re-adjustthe tone curve is judged based on whether or not the same medium as thatin the previously imaged well plate WP is used, and Steps S111 to S113are performed again if necessary.

Note that the tone curve applied to the imaging of each well plate WP isstored as imaging history information in the memory 52 or the storage 53of the host computer 50 while being associated with information foridentifying the well plate WP. As described above, the same processingconditions are desirably applied for the same samples and theinformation of the tone curve applied to each well plate WP can be usedfor this purpose by being stored. This information is given as theimaging information from the host computer 50 to the imaging unit 1according to need when the imaging operation is performed.

<Second Mode>

FIG. 7 is a flow chart showing the second mode of the imaging operationof this imaging system. In this mode, an operation of adjusting the tonecurve differs, but the other operations are the same as in the firstmode shown in FIG. 5. Accordingly, in FIG. 7, the same processings asthose shown in FIG. 5 are denoted by the same reference signs and notdescribed.

In this mode, if the adjustment of the tone curve is judged to benecessary in Step S104, a look-up table corresponding to the medium isselected from a plurality of look-up tables prepared in advance withoutperforming the pre-scanning of the samples. Specifically, the pluralityof look-up tables (LUTs) in which the tone curve represented by thecurve A in FIG. 4 are scaled at different ratios are generated inadvance and stored in the memory 32 of the imaging unit 1 or the memory52 or the storage 53 of the host computer 50. If a data amount is large,these LUTs are preferably stored in the host computer 50.

Further, in the host computer 50, a reference table associating mediuminformation representing the types and amounts of media and the LUTscorresponding to tone curves optimal for the media specified thereby isgenerated in advance in addition to the LUTs for gradation correctionrepresenting the tone curves, and stored in the memory 52 or the storage53.

FIG. 8 is a diagram showing an example of the reference table. In thisreference table 56, the type and the density of the medium, a pouringamount into the well W and the type of the well plate WP are input asthe “medium information”. Since a constant correlativity is thought toexist between the medium specified from these pieces of information andthe luminance value of this medium, the luminance value of the mediumcan be estimated without actual measurement by using the mediuminformation. If the luminance value of the medium can be estimated, theLUT corresponding thereto can be selected.

The medium information is preferably associated with a database used bythe user to manage the samples. Specifically, it has been conventionallywidely practiced to generate a database to be unitarily managed by thehost computer 50 for the contents, preparation conditions and the likeof various samples prepared by the user. If the medium information ofthe samples carried in the well plate WP is given to the imaging unit 1from the database when the well plate WP desired to be imaged by theuser is designated from this database, the user needs not designate themedium information at the time of imaging.

Further, in generating the reference table, it is, for example,considered to experimentally prepare media specified by the mediuminformation in advance, image them to actually measure luminance values,select the LUTs most suitable for these media from the plurality ofprepared LUTs for gradation correction and make them into a database.Further, the compilation of the reference table by the user may beenabled, so that data can be added and deleted.

In an example of FIG. 8, twelve LUTs (numbers of 1 to 12) are assignedone-to-one to twelve sets of the type and density of the medium, thepouring amount into the well W and the type of the well plate WP. If themedium information of the samples to be imaged is known, this referencetable 56 is referred to (Step S121) and one LUT including the tone curveoptimal therefor is selected (Step S122). In this way, as in the firstmode, the tone curve is adjusted to the optimal one corresponding to themedium of the samples and applied to the gradation correctionprocessing.

<Modification>

Note that although the medium information and the LUT number suitabletherefor is associated in the reference table 56 shown in FIG. 8, areference table associating the medium information and the luminancevalues of the media estimated therefrom may be provided instead of this.In this case, an estimated value of the luminance value of the medium isobtained as a result by referring to the reference table based on themedium information. The tone curve may be scaled using this estimatedvalue instead of the actually measured luminance value of the medium Min the first mode and applied to the gradation correction processing.

<Miscellaneous>

FIGS. 9A and 9B are views showing a difference between images due to thepresence or absence of the adjustment of the tone curve. FIG. 9A showsan example of an image obtained by performing the shading correctionprocessing and the gradation correction processing not associated withthe scaling. FIG. 9B shows an example of an image obtained by performingthe shading correction processing and the scaled gradation correctionprocessing. Imaging objects are the same between these examples. Sincethe gradation correction processing of associating the luminance valueof the white reference plate 12 with the maximum gradation value isperformed in the example of FIG. 9A in which the scaling is notperformed, a relative low gradation value is assigned to each pixel andthe image is entirely dark and has a low contrast. On the other hand,since the maximum gradation value is assigned to the luminance value ofthe medium in the example of FIG. 9B in which the scaling is performed,the image is brighter and has a large density difference and a highcontrast since the image is expressed in a wider dynamic range.

Next, a method for preparing samples suitable for imaging using thisimaging system 100 is described. As described thus far, the tone curveis scaled according to the luminance value of the medium. Thus, if themedium differs in each well W, a different tone curve is applied to thegradation correction processing. However, since the original image datagenerated during imaging is successively subjected to the correctionprocessing and deleted from the buffer 32 a, the original image dataneeds to be obtained by individually operating the imaging part 21 toperform imaging for each well W having a different tone curve.

Since this prevents the extension of a time required for the imagingprocess, the samples using the same medium are desirably collected andarranged on the well plate WP.

FIGS. 10A and 10B are views showing an arrangement example of samples onthe well plate. In this example is illustrated the well plate WP inwhich the wells W are arranged in a matrix in eight rows respectivelyassigned with row numbers of A to H and twelve columns respectivelyassigned with column numbers of 1 to 12. As shown in FIG. 10A, it isassumed that cells A are cultured in the wells in the first to thirdcolumns, cells A′ are cultured in the wells in the fourth to sixthcolumns, cells B are cultured in the wells in the seventh to ninthcolumns and cells B′ are cultured in the wells in the tenth to twelfthcolumns. On the other hand, compounds as drugs are added to the mediumat a different concentration in each row. Note that the wells in the rowindicated by the row number B are empty to prevent the mixing(contamination) of drug from the wells (having the row numbers C to H)containing drug to the wells (having the row number A) containing nodrug. In such a well plate WP, sample groups 1 to 32 having differentcombinations of the cell types and the drug concentration are formed.

Since the samples are prepared under the same conditions in each group,a common tone curve can be applied. On the other hand, a different tonecurve may be used for each group among different groups, but actualscanning by the imaging part 21 needs to be performed thirty two timesin this case. If the medium is the same, the same tone curve can beapplied. If a common medium is used in the groups 1 to 16 and anothercommon medium is used in the groups 17 to 32, it is sufficient toperform actual scanning in a region R1 where the groups 1 to 16 arearranged and a region R2 where the groups 17 to 32 are arranged as shownin FIG. 10B. Thus, it is sufficient to perform actual scanning twice. Asjust described, the samples using the same medium are desirablycollected as much as possible and arranged in the well plate WP toefficiently perform the scanning of the imaging part 21 a smaller numberof times.

As described above, in this embodiment, multi-gradation image data isgenerated by performing the gradation correction processing in which theluminance value obtained in the medium is associated with the maximumgradation value in imaging the imaging objects (e.g. cell clusters Sp)carried in the well W of the well plate WP together with the medium M.By doing so, an image can be expressed in multiple gradations in a widedynamic range effectively using the numeral value range of the gradationvalues and an image can be digitized with good image quality.

Specifically, the tone curve in which the luminances from the minimumluminance to the maximum luminance received by the CCD elements 22 areassociated with the gradation values from the minimum gradation value tothe maximum gradation value is stored in advance as the look-up table(LUT) and the tone curve is scaled according to the luminance value ofthe medium and applied to the gradation correction processing, wherebyimaging in such a dynamic range is enabled.

In the first mode of the imaging operation of this imaging system 100,the luminance value of the medium is actually measured by thepre-scanning of the samples and the tone curve is scaled based on thatvalue. Further, in the second mode, the optimal LUT is selected from theplurality of LUTs scaled in advance based on the luminance valueestimated from the medium information specifying the medium. Since thegradation values are assigned in the luminance range corresponding tothe luminance distribution of the original image corresponding to thesamples in each of these modes, an image can be expressed in multiplegradations in a wide dynamic range.

The A/D converter 31 for converting the image imaged by the CCD elements22 into digital data outputs original image data having a bit lengthlonger than 8-bit data finally saved as the multi-gradation image data.Accordingly, the original image data has higher data resolution than themulti-gradation image data. The 8-bit data as the multi-gradation imagedata is generated by performing the gradation correction processing onthe original image data. In such a configuration, the degradation of theimage quality caused by the rounding of lower bits during computationcan be prevented.

Further, in this imaging system 100, the shading correction processingbased on the shading data DS1 obtained from the imaging result of thewhite reference plate 12 is performed together with the above gradationcorrection processing. A variation of the image quality due to asensitivity variation of the imaging optical system can be suppressed bythe shading correction process. Further, images with good image qualitycan be obtained by performing the aforementioned gradation correctionprocessing to correct nonlinearity in the sensitivity of the imagingsystem in addition.

Further, in this imaging system 100, the information indicating whichcorrection characteristic was applied in the gradation correctionprocessing when the samples were imaged is stored as the imaging historyinformation together with the information for specifying the samples.When the samples imaged in the past are imaged again, the gradationcorrection processing to which the gradation correction characteristicapplied in the previous imaging, i.e. last imaging out of the imagingsperformed in the past is applied can be performed regardless of theluminance of the medium at that point of time. Since the correctionprocessing is performed on a plurality of images of the same samplesimaged at time intervals under the same processing conditions, imagessuitable for comparative observation between these images can beprovided.

As described above, in this embodiment, the well plate WP corresponds toa “container” of the invention. Further, the CCD elements 22 of theimaging part 21 and the A/D converter 31 of the control part 30integrally function as an “imager” of the invention, whereas the CPU 33functions as a “data processor” of the invention. Further, theinformation stored in the form of the LUT 32 c corresponds to“correction characteristic information” of the invention, and the memory32 storing this functions as an “information holder” of the invention.Further, the reference table 56 corresponds to a “reference table” ofthe invention. Further, the white reference plate 12 functions as a“luminance reference member” of the invention.

Further, in the imaging operation (FIGS. 5 and 7) of this embodiment,Step S105 corresponds to an “imaging step” of the invention and StepS106 corresponds to a “data processing step” of the invention.

Note that the invention is not limited to the above embodiment andvarious changes other than the above ones can be made without departingfrom the gist thereof. For example, in the first mode of the imagingoperation in the above embodiment, pre-scanning (Step S111) is performedbefore the actual scanning of the samples (Step S105) is performed, andthe tone curve is scaled based on that result to determine the gradationcorrection characteristic. However, imaging for obtaining the luminancevalue of the medium may be performed after actual scanning. Further,even if the luminance value of the medium is obtained from an imageobtained by actual scanning, a result similar to the above can beobtained from the perspective of optimizing the gradation correctioncharacteristic. However, in these cases, the original image dataobtained by actual scanning needs to be temporarily saved since thegradation correction characteristic is not determined at the time ofactual scanning. Thus, a change of providing a large-capacity memory inthe imaging unit 1 or causing the host computer 50 to perform thegradation correction processing or the like is necessary.

Further, the assignment of the functions to the host computer 50 and theimaging unit 1 in the above embodiment is an example and not limited tothe above example. For example, the imaging unit may merely performimaging, i.e. perform only an operation of scanning the samples andgenerating digital data, and all the other data processings may beperformed by the host computer. Further, an integrated imaging systemprovided with all the above processing functions may be adopted.

Further, the imaging part 21 of the above embodiment obtains atwo-dimensional image by scanning and moving the CCD elements 22 as alinear imaging device relative to the imaging objects. However, theimager of the invention may obtain a two-dimensional image, for example,by an area sensor fixedly positioned relative to imaging objects withoutbeing limited to the one obtained by such a scanning movement of alinear sensor.

Further, although the shading correction processing and the gradationcorrection processing are performed on the original image data imaged bythe imaging part 21 in the above embodiment, the gradation correctionprocessing according to the invention is also effective in cases notassociated with the shading correction processing. Further, the shadingcorrection processing may be performed on analog signals outputs fromthe CCD elements 22.

Further, although the calibration process (autofocus adjustmentprocessing and shading processing) are performed after an imaginginstruction from the user is received in the above embodiment, thecalibration process may be automatically performed, for example, afterthe start of the system or regularly without depending on an instructionfrom the user. In such a case, since imaging can be immediatelyperformed when an imaging instruction from the user is received, awaiting time of the user until imaging is finished can be shortened.

Further, in the above embodiment, the scanning movement of the imagingpart 21 relative to the well plate WP is realized by fixing the wellplate WP and moving the light source 29 and the imaging part 21integrally relative to the well plate WP. However, a similar scanningmovement can be realized also by a configuration for fixing the lightsource 29 and the imaging part 21 and moving the well plate WP, and theinvention can be applied also to apparatuses having such aconfiguration. Further, although the light source 29 and the imagingpart 21 are arranged at the opposite sides of the well plate WP carryingthe imaging objects in the above embodiment, the invention can beapplied to apparatuses in which the light source 29 and the imaging part21 are arranged at the same side of the well plate WP and reflectedlight from the well plate WP is read. Further, the imaging objects arenot limited to those carried in the well plate WP as a container andvarious other containers can be used as the “container” of theinvention.

The invention can be particularly suitably applied in fields requiringthe imaging of samples including, for example, biological bodies such aswells on a well plate used, for example, in the fields of medicine andbio-science, and fields of application thereof are not limited tomedical and bio-scientific fields.

Although the invention has been described with reference to specificembodiments, this description is not meant to be construed in a limitingsense. Various modifications of the disclosed embodiment, as well asother embodiments of the present invention, will become apparent topersons skilled in the art upon reference to the description of theinvention. It is therefore contemplated that the appended claims willcover any such modifications or embodiments as fall within the truescope of the invention.

What is claimed is:
 1. An imaging system for imaging a sample in amedium carried in a container as an imaging object, comprising: animager which obtains an original image by imaging the imaging object;and a data processor which generates multi-gradation image data byperforming a gradation correction on the original image, wherein thedata processor associates a luminance value corresponding to a luminanceof the medium in the original image with a maximum gradation value inthe gradation correction.
 2. The imaging system according to claim 1,further comprising an information holder which holds correctioncharacteristic information associating luminance values from a minimumluminance to a maximum luminance and gradation values from a minimumgradation and a maximum gradation, wherein the data processor performsthe gradation correction based on a scaled correction characteristicinformation which is obtained by scaling the correction characteristicinformation such that the luminance of the medium corresponds to themaximum luminance.
 3. The imaging system according to claim 1, whereinthe luminance of the medium is obtained based on the original imageimaged by the imager or an image of the medium imaged by the imagerbefore or after the original image is imaged.
 4. The imaging systemaccording to claim 3, wherein the maximum luminance in the imageobtained by the imager is set as the luminance of the medium.
 5. Theimaging system according to claim 1, further comprising a referencetable associating medium information including a type of the medium anda pouring amount into the container and the luminance of the mediumspecified by the medium information, wherein the luminance of the mediumis obtained based on the medium information given by a user and thereference table.
 6. The imaging system according to claim 1, wherein:the imager outputs data having higher data resolution than themulti-gradation image data as original image data representing theoriginal image; and the data processor performs the gradation correctionon the original image data.
 7. The imaging system according to claim 6,wherein a bit length of the original image data is longer than a bitlength of the multi-gradation image data.
 8. The imaging systemaccording to claim 1, wherein the data processor performs a shadingcorrection based on a shading characteristic obtained from an image of apredetermined luminance reference member imaged by the imager and thegradation correction on the original image.
 9. An imaging method forimaging a sample in a medium carried in a container as an imagingobject, comprising: an imaging step of obtaining an original image byimaging the imaging object; and a data processing step of generatingmulti-gradation image data by performing a gradation correction on theoriginal image, wherein the multi-gradation image data is generated sothat a luminance value corresponding to a luminance of the medium in theoriginal image is associated with a maximum gradation value in thegradation correction of the data processing step.
 10. The imaging methodaccording to claim 9, wherein: correction characteristic informationassociating luminance values of the original image from a minimumluminance to a maximum luminance and gradation values from a minimumgradation and a maximum gradation is generated in advance; and thegradation correction is performed based on a scaled correctioncharacteristic information which is obtained by scaling the correctioncharacteristic information such that the luminance of the mediumcorresponds to the maximum luminance.
 11. The imaging method accordingto claim 9, wherein the luminance of the medium is obtained based on theoriginal image or an image of the medium imaged before or after theoriginal image is imaged.
 12. The imaging method according to claim 9,further comprising an input step of receiving an input of mediuminformation including a type of the medium and a pouring amount into thecontainer from a user, wherein the luminance of the medium is obtainedbased on the medium information and a reference table associated withthe luminance of the medium specified by the medium information.
 13. Theimaging method according to claim 9, further comprising a shadingcorrection step of performing a shading correction on the original imagebased on a shading characteristic obtained from an image obtained byimaging a predetermined luminance reference member.
 14. The imagingmethod according to claim 9, wherein the same gradation correctioncharacteristic as the one applied in the last performed data processingstep for the imaging object is applied in the data processing stepperformed anew when the imaging step and the data processing step areperformed again on the imaging object for which the imaging step and thedata processing state were performed in the past.